Traffic recognition and adaptive ground removal based on LIDAR point cloud statistics

ABSTRACT

An advanced driver assistance system (ADAS) and method for a vehicle utilize a light detection and ranging (LIDAR) system configured to emit laser light pulses and capture reflected laser light pulses collectively forming three-dimensional (3D) LIDAR point cloud data and a controller configured to receive the 3D LIDAR point cloud data divide the 3D LIDAR point cloud data into a plurality of cells corresponding to distinct regions surrounding the vehicle, generate a histogram comprising a calculated height difference between a maximum height and a minimum height in the 3D LIDAR point cloud data for each cell of the plurality of cells, and using the histogram, perform at least one of adaptive ground removal from the 3D LIDAR point cloud data and traffic level recognition.

FIELD

The present application generally relates to vehicle advanced driverassistance systems (ADAS) and, more particularly, to techniques fortraffic recognition and adaptive ground removal based on light detectionand ranging (LIDAR) point cloud data.

BACKGROUND

Some vehicle advanced driver assistance systems (ADAS) utilize lightdetection and ranging (LIDAR) systems to capture information. LIDARsystems emit laser light pulses and capture pulses that are reflectedback by surrounding objects. By analyzing the return times andwavelengths of the reflected pulses, three-dimensional (3D) LIDAR pointclouds are obtained. Each point cloud comprises a plurality of reflectedpulses in a 3D (x/y/z) coordinate system). These point clouds could beused to detect objects (other vehicles, pedestrians, traffic signs,etc.). Ground removal is typically one of the first tasks performed onthe point cloud data, but conventional ground removal techniques do notaccount for non-flat surfaces (curved/banked roads, speed bumps, etc.)or scenes having multiple ground surfaces at different levels.

It is also typically difficult, however, to distinguish betweendifferent types of objects without using extensively trained deep neuralnetworks (DNNs). This requires a substantial amount of labeled trainingdata (e.g., manually annotated point clouds) and also substantialprocessing power, which increases costs. This is also a particularlydifficult task in heavy traffic scenarios where there are many othervehicles nearby. Accordingly, while such ADAS systems work well fortheir intended purpose, there remains a need for improvement in therelevant art.

SUMMARY

According to one example aspect of the invention, an advanced driverassistance system (ADAS) for a vehicle is presented. In one exemplaryimplementation, the ADAS comprises: a light detection and ranging(LIDAR) system configured to emit laser light pulses and capturereflected laser light pulses collectively forming three-dimensional (3D)LIDAR point cloud data and a controller configured to: receive the 3DLIDAR point cloud data, divide the 3D LIDAR point cloud data into aplurality of cells corresponding to distinct regions surrounding thevehicle, generate a histogram comprising a calculated height differencebetween a maximum height and a minimum height in the 3D LIDAR pointcloud data for each cell of the plurality of cells, and using thehistogram, perform at least one of adaptive ground removal from the 3DLIDAR point cloud data and traffic level recognition.

In some implementations, the adaptive ground removal comprisesdetermining a dynamic height threshold indicative of a ground surfacebased on the height differences. In some implementations, the adaptiveground removal further comprises removing or ignoring any 3D LIDAR pointcloud data having a z-coordinate that is less than the dynamic heightthreshold.

In some implementations, the histogram is a feature of a modelclassifier for traffic level recognition. In some implementations, thecontroller is further configured to train the model classifier based onknown traffic level data. In some implementations, the model classifieris a support vector machine (SVM). In some implementations, the trafficlevel recognition comprises using the trained model classifier torecognize a traffic level based on the 3D LIDAR point cloud data. Insome implementations, the controller is further configured to adjust afield of view (FOV) of the LIDAR system based on the recognized trafficlevel. In some implementations, the controller is configured to narrowthe FOV of the LIDAR system for light traffic levels and to widen theFOV of the LIDAR system for heavy traffic levels.

In some implementations, the controller does not utilize a deep neuralnetwork (DNN).

According to another example aspect of the invention, a method ofperforming at least one of adaptive ground removal from 3D LIDAR pointcloud data and traffic level recognition by a vehicle is presented. Inone exemplary implementation, the method comprises: receiving, by acontroller of the vehicle and from a LIDAR system of the vehicle, the 3DLIDAR point cloud data, wherein the 3D LIDAR point cloud datacollectively represents reflected laser light pulses captured by theLIDAR system after the emitting of laser light pulses from the LIDARsystem, dividing, by the controller, the 3D LIDAR point cloud data intoa plurality of cells corresponding to distinct regions surrounding thevehicle, generating, by the controller, a histogram comprising acalculated height difference between a maximum height and a minimumheight in the 3D LIDAR point cloud data for each cell of the pluralityof cells, and using the histogram, performing, by the controller, atleast one of adaptive ground removal from the 3D LIDAR point cloud dataand traffic level recognition.

In some implementations, the adaptive ground removal comprisesdetermining a dynamic height threshold indicative of a ground surfacebased on the height differences. In some implementations, the adaptiveground removal further comprises removing or ignoring any 3D LIDAR pointcloud data having a z-coordinate that is less than the dynamic heightthreshold. In some implementations, the histogram is a feature of amodel classifier for traffic level recognition. In some implementations,the method further comprises training, by the controller, the modelclassifier based on known traffic level data. In some implementations,the model classifier is an SVM. In some implementations, the trafficlevel recognition comprises using the trained model classifier torecognize a traffic level based on the 3D LIDAR point cloud data. Insome implementations, the method further comprises adjusting, by thecontroller, an FOV of the LIDAR system based on the recognized trafficlevel. In some implementations, adjusting the FOV of the LIDAR systemcomprises narrowing the FOV of the LIDAR system for light traffic levelsand widening the FOV of the LIDAR system for heavy traffic levels.

In some implementations, the controller does not utilize a deep neuralnetwork (DNN).

Further areas of applicability of the teachings of the presentdisclosure will become apparent from the detailed description, claimsand the drawings provided hereinafter, wherein like reference numeralsrefer to like features throughout the several views of the drawings. Itshould be understood that the detailed description, including disclosedembodiments and drawings referenced therein, are merely exemplary innature intended for purposes of illustration only and are not intendedto limit the scope of the present disclosure, its application or uses.Thus, variations that do not depart from the gist of the presentdisclosure are intended to be within the scope of the presentdisclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional block diagram of an example vehicle having anadvanced driver assistance system (ADAS) with a light detection andranging (LIDAR) system according to some implementations of the presentdisclosure;

FIG. 2 is a functional block diagram of an example traffic recognitionand adaptive ground removal architecture according to someimplementations of the present disclosure;

FIG. 3 is an example overhead view of a vehicle and a plurality of cellscorresponding to distinct regions surrounding the vehicle for whichheight differences are calculated using LIDAR point cloud data; and

FIG. 4 is a flow diagram of an example method of traffic recognition andadaptive ground removal based on LIDAR point cloud statistics accordingto some implementations of the present disclosure.

DESCRIPTION

As discussed above, there exists a need for improvement in automateddriver assistance (ADAS) systems that utilize light detection andranging (LIDAR) for object detection. It will be appreciated that theterm “ADAS” as used herein includes driver assistance systems (lanekeeping, adaptive cruise control, etc.) as well as partially and fullyautonomous driving systems. A conventional ADAS for object detectionutilizes a deep neural network (DNN) trained by machine learning withtraining data that is annotated (e.g., human labeled) for everydifferent type of object. This requires a substantial amount ofresources, both from a processing standpoint and a labeled training datastandpoint, which increases costs.

In a heavy traffic scenario, for example, there will be a large numberof objects (i.e., other vehicles) present in the three-dimensional LIDARpoint cloud. Having to detect and track each of these objects requiressubstantial processing power. Additionally, the first processing stepfor 3D LIDAR point cloud data is typically to identify and remove (orignore) the ground surface. This is typically performed using a fixedz-coordinate threshold (specific to a particular LIDAR sensor mountingconfiguration) or plane model segmentation. These conventionaltechniques, however, fail to account for non-flat ground surfaces(curved/banked surfaces, speed bumps, etc.) or scenes having multipleground surfaces at different heights.

Accordingly, improved techniques for traffic recognition and adaptiveground removal are presented. These techniques determine LIDAR pointcloud statistics (e.g., data point height differences in variousdistinct cells surrounding the vehicle) and use these statistics toadaptively detect and remove (or ignore) all types of ground surfacesand to recognize different traffic scenarios (heavy traffic, lighttraffic, no traffic, etc.). The adaptive removal of all types of groundsurfaces provides for faster LIDAR point cloud processing and improvedground surface detection and removal accuracy (e.g., reduce false groundsurface detections). The detection of different traffic scenarios can beleveraged to control other ADAS features. For example, a heavy trafficscenario may allow the ADAS features to behave more conservatively, andvice versa. In one exemplary implementation, a field of view (FOV) ofthe LIDAR sensors is adjusted based on the detected traffic scenario.For example, a no traffic or light traffic scenario may allow the LIDARFOV to be tuned for more accurate long distance sensing, which could behelpful for higher speed driving, such as on a highway.

Referring now to FIG. 1, a functional block diagram of an examplevehicle 100 is illustrated. The vehicle 100 comprises a torquegenerating system 104 (an engine, an electric motor, combinationsthereof, etc.) that generates drive torque that is transferred to adriveline 108 via a transmission 112. A controller 116 controlsoperation of the torque generating system 104, such as to generate adesired drive torque based on a driver input via a driver interface 120(a touch display, an accelerator pedal, combinations thereof, etc.). Thevehicle 100 further comprises an ADAS 124 having a LIDAR system 128.While the ADAS 124 is illustrated as being separate from the controller116, it will be appreciated that the ADAS 124 could be incorporated aspart of the controller 116, or the ADAS 124 could have its own separatecontroller. The LIDAR system 128 emits laser light pulses and capturesreflected laser light pulses (from other vehicles, structures, trafficsigns, ground surfaces, etc.) that collectively form captured 3D LIDARpoint cloud data.

Referring now to FIG. 2, a functional block diagram of an exampletraffic level recognition and adaptive ground removal architecture 200is illustrated. It will be appreciated that this architecture 200 couldbe implemented by the ADAS 124 or the controller 116. At 204, the 3DLIDAR point cloud data is captured using the LIDAR system 128. Thiscould include, for example, analyzing return times and wavelengths ofthe reflected laser light pulses. The LIDAR system 128 has a FOV settingthat specifies horizontal and/or vertical scanning angles. This FOVsetting could be adjusted, for example, by the controller 116 via acommand to the LIDAR system 128. A narrower FOV produce higherresolution over a smaller field, whereas a wider FOV produces a lesserresolution over a larger field. The narrower FOV could be particularlyuseful, for example, for long distance scanning, such as during highspeed driving, such as on a highway. The wider FOV, on the other hand,could be particularly useful for other operating scenarios, such as lowspeed driving in crowded environments (parking lots, heavy traffic jams,etc.).

At 208, the 3D LIDAR point cloud data is divided into a plurality ofcells, each cell representing a distinct region surrounding the vehicle100. FIG. 3 illustrates an overhead view 300 of an example plurality ofcells 304 surrounding the vehicle 100. In one exemplary implementation,the angle and radius increment between the cells 304 is 2 degrees and 50centimeters (cm), respectively. It will be appreciated, however, thatany suitable division of the 3D LIDAR point cloud data into theplurality of cells 304 could be utilized. Referring again to FIG. 2, ahistogram is generated at 212. The histogram includes a heightdifference between maximum and minimum heights (z-coordinates) of the 3DLIDAR point cloud data for each respective cell. Very small heightdifferences are likely indicative of that 3D LIDAR point cloud datacorresponding to a ground surface. The histogram is then used to performat least one of adaptive ground removal at 216 and traffic levelrecognition at 220. It will be appreciated, however, that the filtered3D LIDAR point cloud data (after adaptive ground removal) could be usedfor processing tasks other than traffic level recognition, such as, butnot limited to, object detection and tracking.

For adaptive ground removal, the histogram data is analyzed to determinea height threshold indicative of a ground surface. This height thresholdcould be dynamic in that it is repeatedly recalculated for differentscenes. Any points in the 3D LIDAR point cloud data having heights(z-coordinates) less than this height threshold could then be removedfrom the 3D LIDAR point cloud data (thereby obtaining filtered 3D LIDARpoint cloud data) or otherwise ignored. The removal or ignoring of any3D LIDAR point cloud data that is likely a ground surface allows forfaster processing due to the smaller dataset. This also may facilitatethe use of a less expensive controller 116 due to the reduced throughputrequirements. As shown in FIG. 2, one example of the processing of thefiltered 3D LIDAR point cloud is the traffic level recognition at 220.It will be appreciated, however, that the histogram and the unfiltered3D LIDAR point cloud data could also be fed directly to the trafficlevel recognition at 220. The traffic level recognition 220 involvesdetecting a quantity of nearby objects (i.e., other vehicles) based onthe height differences.

In one exemplary implementation, the histogram is a feature of a modelclassifier for traffic level recognition. This model classifier is muchless complex than a DNN as used by conventional methods. One example ofthe model classifier is a support vector machine (SVM), but it will beappreciated that any suitable model classifier could be utilized. Themodel classifier is trained using known traffic level data. This knowntraffic level data could be training histograms that are each labeledwith a traffic level. For example, a binary labeling system could beused where each training histogram is labeled as a “1” (e.g., heavytraffic or a traffic jam) or a “0” (e.g., very light traffic or notraffic). The model classifier is then applied to the filtered (orunfiltered) 3D LIDAR point cloud data to recognize a traffic level nearthe vehicle 100. As previously mentioned, the FOV of the LIDAR system128 could then be adjusted at 224 depending on the recognized trafficlevel.

Referring now to FIG. 4, a flow diagram of a method 400 for performingat least one of adaptive ground removal from 3D LIDAR point cloud dataand traffic level recognition is illustrated. At 404, the controller 116optionally determines whether a set of one or more preconditions aresatisfied. These could include, for example only, the vehicle 100 beingoperated (e.g., the torque generating system 104 being activated) and nomalfunctions being present. At 408, the controller 116 obtains the 3DLIDAR point cloud data using the LIDAR system 128. At 412, thecontroller divides the 3D LIDAR point cloud data into a plurality ofcells (e.g., cells 304) each indicative of a distinct region surroundingthe vehicle 100. At 416, the controller 116 generates a histogramcomprising a calculated height difference between a maximum height and aminimum height in the 3D LIDAR point cloud data for each cell. At 420,the controller 116 determines whether adaptive ground removal is to beperformed. When true, the method 400 proceeds to 424. Otherwise, themethod 400 proceeds to 432.

At 424, the controller 116 determines a dynamic height thresholdindicative of a ground surface based on the height differences in thehistogram. Typically, most of the height differences are within acertain statistical range and therefore can be considered ground cells.At 428, any data points having a height (z-coordinate) less than thisheight threshold are removed from the 3D LIDAR point cloud data (toobtain filtered 3D LIDAR point cloud data) or are otherwise ignored. At432, the controller 116 determines whether traffic level recognition isto be performed. When true, the method 400 proceeds to 436. Otherwise,the method 400 ends or returns to 404. At 436, the controller 116 uses atrained model classifier (e.g., previously trained using the histogramas the model feature) to recognize a traffic level from the filtered (orunfiltered) 3D LIDAR point cloud data. At 440, the controller 116optionally adjusts a FOV of the LIDAR system 128 based on the recognizedtraffic level. This could include, for example, narrowing the FOV for notraffic or light traffic levels and widening the FOV for heavy trafficlevels. The method 400 then ends or returns to 404.

It will be appreciated that the term “controller” as used herein refersto any suitable control device or set of multiple control devices thatis/are configured to perform at least a portion of the techniques of thepresent disclosure. Non-limiting examples include anapplication-specific integrated circuit (ASIC), one or more processorsand a non-transitory memory having instructions stored thereon that,when executed by the one or more processors, cause the controller toperform a set of operations corresponding to at least a portion of thetechniques of the present disclosure. The one or more processors couldbe either a single processor or two or more processors operating in aparallel or distributed architecture.

It should be understood that the mixing and matching of features,elements, methodologies and/or functions between various examples may beexpressly contemplated herein so that one skilled in the art wouldappreciate from the present teachings that features, elements and/orfunctions of one example may be incorporated into another example asappropriate, unless described otherwise above.

What is claimed is:
 1. An advanced driver assistance system (ADAS) for avehicle, the ADAS comprising: a light detection and ranging (LIDAR)system configured to emit laser light pulses and capture reflected laserlight pulses collectively forming three-dimensional (3D) LIDAR pointcloud data; and a controller configured to: receive the 3D LIDAR pointcloud data, divide the 3D LIDAR point cloud data into a plurality ofcells corresponding to distinct regions surrounding the vehicle,generate a histogram comprising a calculated height difference between amaximum height and a minimum height in the 3D LIDAR point cloud data foreach cell of the plurality of cells, and using the histogram, perform atleast one of: i) adaptive ground removal from the 3D LIDAR point clouddata based on a dynamic height threshold indicative of a ground surface,the dynamic height threshold being dynamically determined based on thecalculated height differences of the histogram, and (ii) traffic levelrecognition based on a model classifier having a feature trained usingthe histogram.
 2. The ADAS of claim 1, wherein the adaptive groundremoval further comprises removing or ignoring any 3D LIDAR point clouddata having a z-coordinate that is less than the dynamic heightthreshold.
 3. The ADAS of claim 1, wherein the controller is furtherconfigured to train the model classifier based on known traffic leveldata.
 4. The ADAS of claim 3, wherein the model classifier is a supportvector machine (SVM).
 5. The ADAS of claim 3, wherein the traffic levelrecognition comprises using the trained model classifier to recognize atraffic level based on the 3D LIDAR point cloud data.
 6. The ADAS ofclaim 5, wherein the controller is further configured to adjust a fieldof view (FOV) of the LIDAR system based on the recognized traffic level.7. The ADAS of claim 6, wherein the controller is configured to narrowthe FOV of the LIDAR system for light traffic levels and to widen theFOV of the LIDAR system for heavy traffic levels.
 8. The ADAS of claim1, wherein the controller does not utilize a deep neural network (DNN).9. A method of performing at least one of adaptive ground removal fromthree-dimensional (3D) light detection and ranging (LIDAR) point clouddata and traffic level recognition by a vehicle, the method comprising:receiving, by a controller of the vehicle and from a LIDAR system of thevehicle, the 3D LIDAR point cloud data, wherein the 3D LIDAR point clouddata collectively represents reflected laser light pulses captured bythe LIDAR system after the emitting of laser light pulses from the LIDARsystem; dividing, by the controller, the 3D LIDAR point cloud data intoa plurality of cells corresponding to distinct regions surrounding thevehicle; generating, by the controller, a histogram comprising acalculated height difference between a maximum height and a minimumheight in the 3D LIDAR point cloud data for each cell of the pluralityof cells; and using the histogram, performing, by the controller, atleast one of: (i) adaptive ground removal from the 3D LIDAR point clouddata based on a dynamic height threshold indicative of a ground surface,the dynamic height threshold being dynamically determined based on thecalculated height differences of the histogram, and (ii) traffic levelrecognition based on a model classifier having a feature trained usingthe histogram.
 10. The method of claim 9, wherein the adaptive groundremoval further comprises removing or ignoring any 3D LIDAR point clouddata having a z-coordinate that is less than the dynamic heightthreshold.
 11. The method of claim 9, further comprising training, bythe controller, the model classifier based on known traffic level data.12. The method of claim 11, wherein the model classifier is a supportvector machine (SVM).
 13. The method of claim 11, wherein the trafficlevel recognition comprises using the trained model classifier torecognize a traffic level based on the 3D LIDAR point cloud data. 14.The method of claim 13, further comprising adjusting, by the controller,a field of view (FOV) of the LIDAR system based on the recognizedtraffic level.
 15. The method of claim 14, wherein adjusting the FOV ofthe LIDAR system comprises narrowing the FOV of the LIDAR system forlight traffic levels and widening the FOV of the LIDAR system for heavytraffic levels.
 16. The method of claim 9, wherein the controller doesnot utilize a deep neural network (DNN).